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Abstract 


Space-borne microwave sensors provide critical rain infonnation used in several 
global multi-satellite rain products, which in turn are used for a variety of important 
studies, including landslide forecasting, flash flood warning, data assimilation, climate 
studies, and validation of model forecasts of precipitation. This study employs four years 
(2003-2006) of satellite data to assess the relative perfonnance and skill of SSM/I (F13, 
F14 and FI 5), AMSU-B (N15, N16 and N17), AMSR-E (Aqua) and the TRMM 
Microwave Imager (TMI) in estimating surface rainfall based on direct instantaneous 
comparisons with ground-based rain estimates from Tropical Rainfall Measuring Mission 
(TRMM) Ground Validation (GV) sites at Kwajalein, Republic of the Marshall Islands 
(KWAJ) and Melbourne, Florida (MELB). The relative perfonnance of each of these 
satellite estimates is examined via comparisons with space- and time-coincident GV 
radar-based rain rate estimates. Because underlying surface terrain is known to affect the 
relative perfonnance of the satellite algorithms, the data for MELB was further stratified 
into ocean, land and coast categories using a 0.25° tenain mask. 

Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited 
considerably higher correlations and skills in estimating/observing surface precipitation. 
While SSM/I and AMSU-B exhibited lower correlations and skills for each of the 
different terrain categories, the SSM/I absolute biases trended slightly lower than AMSR- 
E over ocean, where the observations from both emission and scattering channels were 
used in the retrievals. AMSU-B exhibited the least skill relative to GV in all of the 
relevant statistical categories, and an anomalous spike was observed in the probability 
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distribution functions near 1.0 mm hr" 1 . This statistical artifact appears to be related to 
attempts by algorithm developers to include some lighter rain rates, not easily detectable 
by its scatter-only frequencies. AMSU-B, however, agreed well with GV when the 
matching data was analyzed on monthly scales. These results signal developers of global 
rainfall products, such as the TRMM Multi-Satellite Precipitation Analysis (TMPA), and 
the Climate Data Center’s Morphing (CMORPH) technique, that care must be taken 
when incorporating data from these input satellite estimates in order to provide the 
highest quality estimates in their products. 
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1 . Introduction 


Precipitation is a key component of the Earth’s hydrological cycle and an 
important factor in climate change because of its association with the global transport and 
distribution of water and latent heat. Until the advent of meteorological satellites and the 
development of remote sensing techniques for measuring precipitation from space, there 
was no observational system capable of providing accurate estimates of precipitation at 
global scales. Precipitation, moreover, is one of the most difficult physical quantities to 
measure accurately due to extreme variability both temporally and spatially, and 
insufficient observations over the planet’s oceans. 

Since the early 1970s, satellites have been used to quantitatively estimate 
precipitation by observing the emission and scattering processes associated with clouds 
and precipitation in the atmosphere. Multi-channel passive microwave remote sensing 
techniques hold the most promise because these instruments sample rain systems at 
different depths, and subsequently can be used to physically probe precipitation in its 
different thermodynamic states. Since the launch of the single channel Electrically 
Scanning Microwave Radiometer (ESMR) aboard the Nimbus 5 in 1972, many other 
multi-channel microwave platforms have been deployed in space and used to estimate 
surface rainfall over a significant fraction of the earth’s surface. 

However, a single orbiting platfonn does not collect enough rain information to 
accurately estimate precipitation everywhere on the earth’s surface. The Tropical Rainfall 
Measuring Mission (TRMM) TRMM Microwave Imager (TMI), United States Defense 
Department’s Special Sensor Microwave/Imager (SSM/I), National Oceanic and 
Atmospheric Administration (NOAA) Advanced Microwave Scanning Radiometer- EOS 
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(AMSR-E) and Advanced Microwave Sounding Unit (AMSU), for example, each 
provide about two samples per day from any localized region of the earth’s surface. 
Statistical sampling on regional scales can thus be significantly improved by combining 
the rain information from a multitude of satellites. These passive microwave satellite 
platfonns are currently being used to develop high-resolution global estimates of 
precipitation using varying types of merging of the various satellite observations, 
including the TRMM Multi-Satellite Precipitation Analysis (TMPA) (Huffman et al. 
2007) the Climate Data Center’s Morphed (CMORPH) analysis (Joyce et al. 2004) and 
the Precipitation Estimation from Remotely Sensed Information using Artificial Neural 
Networks (PERSIANN, Hsu et al. 1997, Sorooshian et al. 2000), all of which provide 
near-global 3-hourly estimates at 0.25° resolution. Identifying the strengths and 
weaknesses of the satellite measurements associated with each sensor class is critical to 
understanding the measurement and error characteristics of the multi-satellite products 
used in various applications, including climate modeling. 

Rain estimates determined from ground-based sensors provide an independent 
source of validation for inferring the error characteristics of estimates derived from 
space-bome remote sensors. However, to eliminate the temporal sampling component of 
the error, it is necessary to match both the satellite and GV estimates in time and space. 
In previous studies, Ferraro and Marks (1995) estimated errors in SSM/I/ inferred rain 
rates by matching SSM/I data obtained from F8 and Fll to radar estimates collected in 
the United States, the United Kingdom and Japan. Lin and Hou (2009) validated rain 
estimates from eight microwave sensors (TMI, F13, FI 4, F15, AMSR-E, N15, N16, and 
N17) by comparing near-coincident measurements from the TRMM Precipitation Radar 
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(PR). 


The Tropical Rainfall Measuring Mission (TRMM) satellite program (Simpson et 
al. 1988, Simpson et al. 1998, Kummerow et al. 2000) has been operational since its 
launch in November 1997. As part of the TRMM program, a Ground Validation (GV) 
program was established in order to provide validation of the TRMM satellite estimates. 
The GV program established four principal sites that use both ground-based radars, and 
networks of rain gauges and disdrometers to provide high-resolution rain rate 
measurements, both temporally and spatially, for such efforts. Wolff et al. (2005) 
provide an overview of the TRMM GV program. Wolff & Fisher (2008) used TRMM 
GV data to show the relative accuracy of the three principal TRMM satellite retrievals 
derived from the PR and the TMI and Combined (COM) algorithms. Significant effort 
has been made towards improving the quality of the TRMM GV rain rate estimates. In 
particular, success in post-calibrating the radar data from Kwajalein, Republic of the 
Marshall Islands (Silberstein et al. 2009), has provided a high-quality, open ocean data 
set for comparison to TRMM and other satellites precipitation retrievals (Marks et al. 
2009). 

In this study, we use four years (2003-2006) of GV data from Kwajalein, 
Republic of the Marshall Islands (KWAJ) and Melbourne, Florida (MELB), to assess the 
relative performance of satellite precipitation estimates from seven polar-orbiting 
satellites and the TRMM Microwave Imager (TMI), which flies in sun-asynchronous 
orbit. The comparisons are based on instantaneous estimates, all averaged to 0.25°. 
Using GV as a reference, we will show both the similarities of the various satellite 
estimates, as well as some fundamental differences, due to algorithmic and instrumental 
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issues, as well as beam-filling issues, given the different footprint sizes of the various 
satellite platforms. Although considerable effort has been made to produce the highest 
quality and most accurate rain rate data from the GV observations, no claim as to the 
absolute accuracy of the GV precipitation estimates can be made, but rather offer GV 
data as a reference, from which the relative performance of the satellite estimates can be 
made. 

2. Satellite microwave radiometers: SSM/I, AMSU-B, AMSR-E and TMI 

a. Data Sources: General 

This study analyzes data collected from multi-channel microwave rain sensors on 
board eight space-bome satellites. The rain sensors involved in this study include AMSU- 
B on N15, N16, and N17 satellites, SSM/I on F13, F14 and F15 satellites, AMSR-E on 
the Aqua satellite, and the TMI on the TRMM satellite. Each platform samples 
precipitating cloud structure across a broad range of discrete frequency bands in the 
microwave spectrum. Table 1 shows the frequency, polarization, and beam width of the 
various channels for each sensor class. 

Level II AMSR-E data and AMSU-B data were acquired from the National Snow 
and Ice Data Center (NSIDC) in their native resolution. This data was then gridded to a 
resolution of 0.25 degrees using software developed by Eric Nelkin at the Goddard Space 
Flight Center. SSM/I were obtained from the Goddard Earth Science Distributed Active 
Archive Center (DAAC), which was already gridded at 0.25°. The TMI data was gridded 
to 0.25 degrees by the TRMM Science Data and Information System (TSDIS) for use in 
this study, at the special request of the authors of this study. (E. Stocker, personal 
communication, 2008). 
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b. SSM/I, TMI and AMSR-E 


The SSM/I, TMI and AMSR-E are conically scanning microwave radiometers 
with similar physical characteristics. The SSM/I were designed for the U.S. Air Force 
Defense Meteorological Satellite Program (DMSP), which began in June 1987 with the 
successful launch of F8. SSM/I data for this study were obtained from FI 3, F14 and F15, 
which were launched in 1995, 1997 and 1999, respectively. The DMSP satellites fly in a 
near circular polar orbit at a nominal altitude of 865 km and an orbital period of about 
102 minutes. 

The SSM/I is equipped with seven linearly polarized channels, which sample 
microwave radiances at four different frequencies over a swath of about 1400 km. Dually 
polarized measurements - horizontal and vertical - are collected at 19.35, 37.0 and 85.5 
GHz, while the channel centered on the water vapor line at 22.24 GHz measures only 
vertically polarized radiances (Prabhakara 1992, Kummerow and Giglio 1994). 

The TRMM satellite was launched in November 1997 with a payload that 
included both passive and active rain sensors: a microwave sensor (TMI) and the 
precipitation radar (PR). The TMI was based on the original design of the SSM/I, with a 
few additions and modifications: the TMI added two dually polarized emission channels 
at 10.7 GHz, while the 21.3 GHz channel on the TMI was shifted slightly off center of 
the water vapor line to avoid saturation (Kummerow 1998). Both the TMI and SSM/I 
only allocate one vertically polarized channel to the water vapor band at 22.3 GHz. 

TRMM flies in a sun-asynchronous, low-earth orbit, with an inclination angle of 
35 degrees and a mean altitude of 402 km. Because TRMM flies in a lower earth orbit 
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than the other satellites, the TMI covers a smaller swath of 878 km and has a smaller 
footprint at each frequency. The TRMM satellite, unlike the polar orbiters, which collect 
two samples per day at about the same time, precesses through the entire diurnal cycle 
over a period of about 46 days. 

AMSR-E is flown onboard the Aqua satellite and is part of the multi- satellite 
Earth Observing System (EOS). Aqua was launched in May 2002, and orbits at an 
altitude of 705 km with an orbital period of about 99 minutes. AMSR-E has 12 dually 
polarized channels that sample microwave radiances at six different frequencies ranging 
from 6.9 to 89 GHz across a swath of about 1445 km (see Tables 1 and 2). AMSR-E is 
similar in design to the TMI, but includes two additional channels at 6.9 GHz, however 
poor spatial resolution limits the usefulness of these channels in rainfall estimation. 
AMSR-E also includes a horizontally polarized channel at 23.8 GHz, with the off-center 
shift from the water vapor line towards a slightly higher frequency. 

Rain rate retrievals for the SSM/I, TMI AMSR-E were carried out using Version 
6 of the Goddard Profiling (GPROF) algorithm described by Kummerow et al. (1996), 
Kummerow et al. (2001), Kummerow et al. (2006), Wilheit et al. (2003) and others, 
although it should be noted that minor modifications of the basic Version 6 algorithm are 
employed by the algorithm developers for each platform. However, in general, the 
GPROF algorithm retrieves both instantaneous rainfall and the vertical latent heating 
structure by constructing a vertical profile from the observed brightness temperatures in 
each channel. Rain rates are calculated using a Bayesian inversion procedure, which 
relates the retrieved profile, Tb, to a simulated rain profile R: 

Pr(R|r,)ocPr(R)*Pr(r,|R) m 
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Given the observed profile Tb, Eqn. 1 relates the conditional probability of observing a 
particular profile R to the product of the two probabilities on the right hand side. The first 
term is derived from cloud-resolving models and describes the probability that a certain 
rain rate profile R will be observed. The second term is obtained by matching radiative 
transfer calculations (corresponding to the cloud-resolving model rainfall 
profiles) and describes the Bayesian probability of observing the brightness temperature 
Tb given the rain rate profile R (Kummerow et al. 2001,Wilheit et al. 2003). 

The estimation of surface rain rates from space-borne microwave sensors depends 
on the ability to separate emission and scattering radiances due to rain and clouds from 
the radiative upwelling emanating from the earth’s surface (Weinman and Guetter 1977). 
The task is complicated by the fact that radiative properties of the land and ocean 
surfaces differ significantly. Whereas the ocean surface is radiometrically cold and 
homogeneous, the land surface is radiometrically warm, and emissivities over land can be 
highly variable in time and space. Coastal regions, which include radiative contributions 
from both ocean and land, require even more complicated “physics” and assumptions and 
yield the most uncertain results. 

Consequently, GPROF handles the calculation of rain rates differently depending 
on the underlying terrain. First, each pixel sampled is classified as ocean, land or coast 
using a fixed surface-terrain mask that was originally developed at 1/6° degree resolution 
for the TRMM TMI. For KWAJ and MELB, the surface-terrain masks used in this study 
are shown in Figs. 2a and 2b, respectively. The “ocean” algorithm derives a vertical 
profile of precipitation using all of the brightness temperature infonnation collected from 
the available channels, and thus more physically based because the low frequency 
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emission channels more directly probe the precipitable water at the cloud base. The land 
and coast algorithms, on the other hand, are empirically based because they rely on the 
high frequency scattering channels (85.5 GHz for SSM/I and TMI and 89.0 GHz for 
AMSR-E), which are correlated with ice scattering processes above the freezing level in 
the cloud. Over land and coast the emission spectrum from the surface is too warm and 
highly variable to distinguish from the rain signal in the atmosphere. 

The GPROF land algorithm is based on the National Environmental Satellite Data 
and Information Service (NESDIS) approach. A more complete description of the land 
algorithm can be found in Ferraro (1997) and McCollum and Ferraro (2003). The 
operational coast algorithm was originally developed from the Goddard Scattering 
Algorithm (GSCAT) and is described in McCollum and Ferraro (2005). Again, minor 
differences in the various GPROF algorithms employed by the various platforms are 
common. 

c. AMSU 

The Advanced Microwave Sounding Unit (AMSU) is a cross-track, line scanning 
observational system consisting of two multi-channel radiometers: AMSU-A and AMSU- 
B. The AMSU-B sensor package was designed for probing the temperature and moisture 
structure of the atmosphere. It is also used being in the estimation of surface rainfall. This 
sensor package is flown aboard the NOAA-15 (N15), NOAA-16 (N16) and NOAA-17 
(N17) satellites at an altitude of 850 km. (swath 1650 km) 

AMSU-A has fifteen channels that range from 23.8 to 89.0 GHz and AMSU-B 
has 5 channels ranging from 89.0 to 183.31 GHz. The AMSU-B rain rate algorithm is 
fundamentally based on the indirect correlation between surface rain rate (RR) and ice 
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scattering aloft as observed by the high frequency channels (Spencer et al 1989, Weng et 
a. 2003, Qiu et al. 2005). The algorithm simultaneously derives the ice water path (IWP) 
and the effective particle diameter D e . Like GPROF, the AMSU algorithm discriminates 
between ocean, land and coastal terrains. In each case, the IWP is obtained using a two- 
stream approximation to the radiative transfer equation that uses the 89 and 150 channels 
(Kongoli et al. 2007, Vila et al. 2007). 

Recent algorithmic improvements use information in some of the lower frequency 
channels, namely, 23.8, 31.4, 53.8 and 183 GHz to refine the rain estimate for a particular 
case (ocean, land coast). The 23.8 and 31.4 GHz channels are used to increase the areal 
coverage over the oceans, while the 53.8 GHz channel is used identify falling snow. The 
three channels at 183 GHz, centered on the strongly absorbing water vapor band, is used 
to compute a convective index to discriminate between convective and non-convective 
rain systems (Kongoli et al. 2007, Vila et al. 2007). 

Rain rates are determined based on a non-linear relationship between RR and IWP 
computed using the fifth generation Mesoscale Model (MM5) developed by Penn State 
University and the National Center for Atmospheric Research (NCAR). The mean IWP- 
RR relation can be statistically determined and assumes the form 

RR = a 0 + alWP +a 2 IWP 2 (2), 

where ao, ai and estimated constants. 

Multiple scattering channels are also needed in order to simultaneously determine 
both IWP and D e without ambiguity (Weng et al 2003). Although AMSU has some 
distinct weaknesses due to its underutilization of the lower frequency channels, its use of 
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the high frequency channels offers a smaller field of view for resolving, while the 
addition 150 GHz channel allows for the detection of lighter rain rates. 

d. Inter-sensor sources of uncertainty 

Several different sources of uncertainty need to be accounted for in explaining the 
inter-sensor differences in the passive microwave rain retrievals examined in this study. 
Variations in the rain retrievals can be attributed to differences in a) precipitation 
algorithms; b) Field-of-View (FOV), c) instrument characteristics; d) sampling and e) 
calibration. These sources of uncertainty not only explain some of the observed 
differences between the rain sensors but also address the absolute uncertainties as well. 
Here we are more concerned with relative inter-sensor differences, both with respect to 
the space-borne sensors and GV. Moreover, the relative contribution of these 
uncertainties in explaining the observed inter-sensor differences depends on rudimentary 
factors such as the underlying surface (ocean, land or coast) and the type of rain system. 
For example, over the oceans the large FOV in the low frequency emission channels 
leads to so-called “beam filling” problem, resulting from an inhomogeneous distribution 
of rain infonnation across the FOV in association with the existence of strong rainfall 
gradients, and/or regions of rain and no rain within a single footprint (Joyce et al. 2004). 
If not corrected for, smearing of the signal across the FOV simply produces a 
homogenous FOV, leading to an underestimate of the rainfall due to non-linear response 
of brightness temperature to rain rate (Kummerow 1998). A larger FOV both exacerbates 
the beam-filling problem and also reduces the number of FOVs, i.e., samples, in a given 
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0.25 grid box. The beam-filling problem tends to represent the most significant source of 
error for the TMI, AMSR-E and SSM/I over the oceans. Of course, AMSU faces a 
different problem over the oceans, where rain rate is inferred entirely from an ice 
scattering algorithm that relies solely on the precipitation signal in the high frequency 
channels. The AMSU algorithm thus has difficulty observing low rain rates (< 1 mm hr" 

' ) and warm rain processes, where no ice scattering is present. 

The FOV depends on the altitude of satellite, the observing frequency and the 
beam width of the antenna. Table 1 provides the beam width of the various channels for 
the different satellite platfonns used in this study. Table 2 shows the altitude and FOV 
area sizes for three different frequencies for each of the satellite platforms. Although the 
TMI and AMSR-E fly at different altitudes, they have about the FOV because the 
antenna beam width on AMSR-E is significantly narrower than TMI. For example, the 
TMI beam width is 0.42 and 0.43 for the vertical and horizontal 85.5 GHz channels, 
respectively, while the AMSR-E beam width is 0.18 for both the vertical and horizontal 
89.0 GHz channels. 

Over land and coast, GPROF is limited to the high frequency channels due to 
ground contamination in the low frequency emission channels. The low frequency 
emission channels are subsequently not utilized. As shown in Table 2, the high frequency 
channels have a significantly smaller FOV, however, the ice scattering processes 
responsible for producing a rain signature occur at higher elevations and therefore are 
less correlated with surface rainfall. It is also important to note that even though the 
GPROF rain algorithm generates rain rates for the SSM/I, TMI and AMSR-E retrievals, 
as was already shown, each of these sensors has different instrument characteristics 
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related to the number, frequency and polarization of the different channels. Consequently, 
though all three sensors are similar in design, they are not entirely equivalent regarding 
their respective measurements. 

3. GV Data 

The GV data for this study used the official TRMM 2A-53 rain products for 
KWAJ and MELB produced by the TRMM Satellite Validation Office and available 
from the Goddard Earth Sciences Data and Information Services Center (GES DISC). 
The 2A-53 product provides instantaneous rain rates at 2 km x 2 km horizontal 
resolution, extending 150 km from the respective GV radar. The GV radar data was 
processed using official Version 5 (MELB) and Version 7 (KWAJ). The algorithm 
applies the Window Probability Matching Method (Rosenfeld 1995) to the statistical 
determination of rain rates from radar reflectivities. The GV processing system and 
descriptions of the GV algorithms are detailed in Wolff et al. 2005. We note that the 
KWAJ data, now Version 7, has undergone significant improvement from previous 
versions, after the application of the Relative Calibration Adjustment (RCA) technique 
was applied to correct calibration and pointing angle errors in the raw reflectivity data 
(Silberstein et al. 2009). The RCA method and its application to the KWAJ reflectivity 
data are described in Marks et al. 2009. Marks et al. convincingly demonstrate that the 
year-to-year radar-rainfall relationships converge, as was to be expected from long-term 
analysis of drop size distributions. Further, Marks et al. 2009 show that the mean 
difference between the KWAJ reflectivity estimates to be within ± 1 dB of the well 
calibrated and stable TRMM Precipitation Radar (PR) (Takahashi et al. 2003). 
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4. Data Analysis: Description 

This study assesses the relative performance of the eight microwave sensors for 
the GV sites at KWAJ and MELB over a four-year period from 2003 to 2006. The 
validation strategy was to inter-compare ground and space based measurements by 
matching the retrieved instantaneous rain rates from each sensor to the GV rain rates 
during satellite overpasses of the GV site. The matched rain rates were averaged at a grid- 
resolution of 0.25° x 0.25° (latitude x longitude). The key advantage of matching the 
satellite and GV data sets at instantaneous scales is that the temporal sampling error 
associated with non-continuous regional sampling by the orbiting satellites can be 
eliminated as a major source of uncertainty. Although there are myriad problem with 
estimating precipitation with radar data, such as anomalous propagation, ground clutter, 
beam filling, range effects, improper rain/reflectivity relationships etc., the authors of this 
study suggest that rather than rely too heavily on simple statistics such as biases, which 
might occur from over- or under-estimation by the radar, the satellite or both, one should 
make these comparisons using a variety of statistical tests in order to determine how the 
relative performance of the various satellite estimates vary. This validation strategy uses 
the ground data as an empirical reference for assessing the relative performance of 
passive microwave-based rain retrievals at instantaneous scales. 

The PMM method used to generate radar-derived rain rates from measured 
reflectivity depends on the instantaneous rain rate information from an extended network 
of gauges as a function of range. Figure la and lb show the radar and gauge networks for 
KWAJ and MELB, respectively. The gauge network at KWAJ only extends out to about 
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the 100 km range ring shown in Fig la, whereas in Fig. lb, it can be seen that the gauge 
network at MELB extends well beyond the outer 150 km range ring. In the case of 
KWAJ, satellite rain rates were only matched out to 100 km but were extended out to 150 
km in the case of MELB. 

Each 0.25° grid box, as mentioned in the previous section, was classified as land, 
coast or ocean according to a surface terrain mask. The original TRMM terrain map was 
produced at a scale of 1/6°, whereas the matching in the study was at 0.25°. For 
classification purposes, the mask was interpolated to 0.25°. For cases where more than 
one terrain type existed, the classification was determined based on a “majority rules” 
criterion. The 0.25° classifications for KWAJ and MELB are shown in Fig. 2a and 2b, 
respectively. KWAJ is a pure, open ocean site and as seen in Fig. 2a, every 0.25 grid box 
is classified as ocean. MELB, on the other hand, contains all three classifications, with 
about 1/3 of the boxes classified as coast 

The seven polar orbiting satellites and TRMM differ fundamentally in their 
temporal sampling of the atmosphere. For a given grid box, each of the polar orbiters 
collect two samples per day at the about the same nominal times, and thus mean rain rate 
inferred from these observations incurs an intrinsic diurnal bias. TRMM on the other 
hand, reduces the diurnal bias by precessing through the diurnal cycle in a period of about 
46 days. Thus, over the four-year period employed in this study, TRMM is able to well 
capture the diurnal cycle at both KWAJ and MELB. To illustrate the potential impact of 
the diurnal cycle on longer-term rainfall estimation, estimated diurnal cycles were 

generated for KWAJ and MELB based on mean hourly conditional mean (R \ R > 0) rain 
rate computed from five years (2000-2004) of radar data. These results are shown in Fig. 
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3a-b. The filled circles displayed in each profile designate the nominal times when each 
satellite flies over the site. Fig. 3b for MELB reveals a strong diurnal cycle, 
characterized by its high amplitude maximum in the late afternoon (~3 pm LST), and a 
relative minima occurring between 1 1 pm - 6 am. KWAJ, on the other hand, exhibits a 
much weaker diurnal cycle, with a nocturnal maximum occurring at around 0100-0200 
LST, and a minimum during the daytime hours (Wolff et ah, 1995), typical of a tropical 
oceanic environment. 

a. Probability Distributions by Occurrence and Rain Volume 

This section analyzes the probability distribution functions (PDF) of rain rates for 
the satellite and co-incident GV data during the study period. Two PDF’s were computed 
for each satellite: (1) PDF by occurrence (PDF C ) and (2) PDF by rain volume (PDF V ). 

The PDF C provides statistical information on the breadth and shape of the rain rate 
distribution and highlights differences in the respective estimate’s sensitivity as a 
function of rain rate. The PDF V is a normalized distribution obtained by multiplying the 
PDFc(bin) by the rain rate assigned to each bin, and represents the percentage of rain 
accumulation that each rain rate interval contributes to the total rain volume. We note 
that these distributions are computed using collocated data in both space and time. In 
other words, the GV estimates that go into the distributions are only those that are 
sampled during a given satellite overpass. 

PDF C and PDF V for KWAJ are shown in Fig. 4. All of the satellite rain rates 
determined from the GPROF algorithm (AMSR-E, F13,F14, F15 and TMI) exhibit 
similar PDF V . AMSR-E tends to slightly underestimate heavier rain rates, relative to GV. 
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Both TMI and AMSR-E distributions agree quite well, especially in their ability to detect 
lighter rain rates, which all of the SSM/I and AMSU-B estimates fail to detect. Further 
the PDF V for SSM/I also show decreased performance in detecting moderate rain rates 
(above about 0.5 mm hr’ 1 ), and a tendency to overestimate the highest rain rates. The 
AMSU-B rain distributions are highly peaked, around 1 mm hr’ 1 , which is due to ad hoc 
adjustments to the product algorithm by its developers in an effort to increase the 
sensitivity of the AMSU-B estimates in light rain, which is significantly diminished by 
the lack of emission channels available on the platforms (R. Ferraro, personal 
communication, 2008). The result shows a large peak in rates at about 1 mm hr’ 1 , which 
we suggest is not physically plausible and should be considered when using AMSU-B 
data over oceanic areas. 

Figures 5a-d show the PDF C and PDF V for GV and satellite estimates at MELB 
over “Ocean”, “Land”, “Coast” and “All” terrain types (see Fig. 2b for reference), 
respectively. Fig. 5a shows the respective PDFs over ocean at MELB. It is obvious that 
these PDFs are quite similar to the KWAJ PDFs, indicating consistency in our results. 
Again the AMSR-E distributions are quite similar to the GV estimates, while both SSM/I 
and AMSU-B estimates tend to under-sample the lightest rain rates. The peaked AMSU- 
B distributions are again evident, at about 1 mm hr’ 1 , again due to the lack of emission 
channels for the platform. 

Fig. 5b shows the respective PDFs over land areas at MELB. Both TMI and 
AMSR-E show some diminished performance in detecting the lightest rain rates, relative 
to GV. At the highest rain rates, AMSR-E and SSM/I (except FI 5) all agree fairly well 
with GV estimates, while TMI slightly overestimates the heaviest rain rates. Not 
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surprisingly, the AMSU-B estimates over land are significantly better than over ocean. 
Indeed, the AMSU-B estimates are in line with the TMI and AMSR estimates, and 
somewhat better than the and SSM/I estimates, which is likely due to the additional 
scattering channels available for the AMSU-B platform. Fig. 5c shows the respective 
PDFs over coastal areas at MELB. Both TMI and AMSR-E show similar PDFs relative to 
GV, which is somewhat surprising in that Wolff and Fisher (2008) demonstrated 
significant differences between the GV and TMI distributions, at the scale of the TMI- 
footprint (approximately 154 km 2 ). The AMSU-B estimates also perform quite well, 
relative to GV, over coastal areas. Fig. 5d shows the respective PDFs over the entire GV 
domain at MELB. Once again, the AMSU-B distributions most closely match the GV 
distributions, and show increased skill in detecting the lighter rain rates. The SSM/I 
distributions, show a large peak at the more moderate rain rates, not seen in GV, but 
consistent with coastal influences shown previously by Wolff & Fisher (2008). The 
AMSU-B estimates over the entire domain perform relatively well, with a strong peak in 
the distributions introduced by the erroneous ocean- area estimates. 

b. Monthly Mean Rain Rates 

In this section, instantaneous satellite rain rates were validated on monthly scales 
by computing monthly rainfall estimates using the matched data sets from KWAJ and 
MELB at 0.25°. The results for KWAJ and MELB are shown in Figures 6 and 7, 
respectively. Also, Table 3 and 4 provide summary statistics of our comparisons for 
KWAJ and MELB, respectively. The mean monthly rain estimates for KWAJ and MELB 
show good agreement with measurements made from earth-based and space-based 
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microwave sensors, but there are also some important differences. 

In Fig. 6a for KWAJ, all eight satellites are highly correlated with GV month-to- 
month and demonstrate good skill at estimating surface rainfall on monthly time scales. 
However, as displayed in Table 3, all eight satellites show an overall negative bias 
compared to GV. Similar results are observed in the MELB ocean case shown in Fig. 7a. 
With the exception of FI 5 (+0.5%), all of the other sensors exhibited a negative bias. The 
GV-Satellite comparisons over MELB land exhibit a greater tendency for the satellite 
sensor to overestimate the rainfall during the summer months. This tendency is especially 
prevalent in the SSM/I group, which resulted in overall biases ranging between 13.1% 
and 25.5%. Furthermore, the bias exhibits oscillatory behavior, overestimating 
convective summertime rainfall, while underestimating wintertime rainfall. The MELB 
Coast case displays similar features as observed in the Land case, with SSM/I exhibiting 
the highest summertime biases. For MELB Coast, the SSM/I biases ranged from 44.4% 
to 71.1%. 

The AMSU-B sensors showed less consistency as a group. For MELB coast, N15 
tends to overestimate summertime rainfall over land and coast, resulting in an overall 
positive bias in both cases. The N15 bias in the coast case is only 0.5% because of the 
tendency for N15 to underestimate surface rainfall during the transition months 
associated with spring and fall. N17, on the other hand, was negatively biased for both 
land (-18.3%) and coast (-42.9%). Note that in Fig. 3b, N17 straddles the beginning and 
end of the convective period, characterized by the large amplitude that peaks in the mid- 
afternoon. 

The results for MELB considered alongside the results for KWAJ illustrate some of 
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the issues over land and coast, and the importance of the lower frequency channels in 
accurately determining instantaneous surface precipitation from convective systems. 
During the summer months, Central Florida is dominated by smaller-scale sea breeze 
convection, which can produce large anvils. The GPROF land and coast algorithm must 
first screen the FOV to determine whether or not rain exists before attempting to quantify 
the rainfall based on the 85.5/89.0 GHz rain signature. Non-precipitating anvils can easily 
be interpreted as rain due to their ice content. Moreover, in the land case, the rain 
algorithm has been "engineered" to match cloud profiles to brightness temperature based 
only on the high frequency channel (Wang et al. 2009). Consequently, the algorithm does 
not yet address the actual microphysics, whereas in the ocean case, the complex 
microphysics is treated more realistically by constructing a Tb vector from all of the 
different emission and scattering channels. The cloud profiles in the ocean case can 
therefore more adequately represent the actual microphysics based on a radiative 
sounding through the column. Since biases over land and coast can depend on regional 
factors, more study is needed to better probe and understand this physical nature of the 
bias. 


c. Correlations and Regressions 

Scatter-plots of the various satellite estimates versus GV estimates are shown in 
Figs. 8 and 9, for KWAJ and MELB, respectively. These plots provide the regression 
equations and correlations. The slope of each regression equation provides an indication 
of the relative biases between the various estimates. Fig. 8 shows the scatter-plots for 
KWAJ. The AMSR-E estimates exhibit the highest correlation of 0.89, slightly greater 
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than the TMI, which shows a correlation of 0.86. However, with a slope of 0.79, the TMI 
exhibits a slightly lower bias than AMSR-E, which had a slope of 0.75. The SSM/I and 
AMSU-B correlations are significantly lower, ranging from 0.62 to 0.67. The slopes of 
the SSM/I range from 0.68 to 0.78, while the AMSU-B slopes range from 0.49 to 0.65. 

Fig. 9a shows the scatter-plots for MELB over ocean areas only. AMSR-E 
estimates are better matched with GV estimates with a correlation of 0.86, and a slope of 
0.74, consistent with our KWAJ results. TMI correlations are 0.84 with a slope of 0.64. 
The correlations for the SSM/I rain rates fell within a tight range between 0.66-0.72, and 
though lower than TMI and AMSR-E, were significantly greater than the AMSU-B, 
whose correlations fell within a range between 0.52-0.54. These results for MELB ocean 
are again consistent with the results for KWAJ. 

Fig. 9b shows the scatter-plots for MELB over land areas only. Here, TMI show 
the highest correlation (0.74) with a slope of 0.86. AMSR-E estimates show a 
correlation of 0.72, but with a decreased slope of 0.62, indicating an increased bias 
(AMSR-E lower than GV) than over land areas. However, it is important to note that 
neither SSM/I nor AMSU-B estimates provided significantly improved skill over land 
areas. 

Fig. 9c shows the scatter-plots for MELB over coastal areas only. Although none 
of the estimates provide correlations greater than 0.74 (TMI), AMSR-E and SSM/I/F15 
both have correlations of 0.69. Somewhat surprisingly, the slopes of the SSM/I 
regression lines are all near unity (0.91 to 1.05). AMSU-B fares worse than either 
AMSR-E or SSM/I, with slopes ranging from 0.34 to 0.48 and correlations on the order 
of 0.5. 
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Fig. 9d shows the scatter-plots over the entire GV domain at MELB. TMI shows 
the highest correlation of 0.74, with a slope of 0.71, while AMSR-E shows a correlation 
of 0.72 and a slope of 0.67. SSM/I estimates show correlations ranging from 0.64 to 
0.68, with slopes ranging from 0.81 to 0.93. AMSU-B estimates show the lowest 
correlations, ranging from 0.50 to 0.55, with slopes ranging from 0.37 to 0.50. 

<7. Rain Rate Profiles 

Mean instantaneous satellite rain rate profiles were constructed for KWAJ and 
MELB by sorting the satellite rain rates along the GV rain rate continuum. For this 
analysis, the matched GV-satellite rain rates were averaged at logarithmic binning 
intervals between 0 and 12 mm hr’ 1 according to the value of the GV rain rate. Wolff and 
Fisher (2008) previously applied this analysis scheme to the validation of TRMM Level 
II rain rates. In that study, GV rain rates were matched to rain rates obtained from the 
TMI, PR and Combined products at the scale of the TMI footprint. The GV-satellite data 
were then binned and averaged on a linear scale at 1 mm hr" 1 intervals. 

When analyzed on a linear scale the highest rain rates are under-sampled, while 
the lowest rain rates are over-sampled. The logarithmic scale, applied to this case, has the 
advantage of naturally partitioning the data into sampling bins of near equal size. The 
logarithmic scaling provides considerably more resolution at the low end of the spectrum 
where the sampling was best. How each sensor performs at the low and high ends of the 
rain rate spectrum are important criteria in evaluating the relative performance and 
uncertainty of each sensor class. The range of each profile was constrained at 12 mm hr’ 1 , 
which represents about 99% of the total rain volume for each sensor. 
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The rain rate profiles for all eight satellites are shown in Figs. 10 and 1 1. For the 
case of MELB, the matched data was further stratified into ocean, land and coast regimes. 
These are displayed in the four panels shown in Fig. 11. In this evaluation, we examine 
the members of each sensor class as a group (AMSU-B and SSM/I), since each class 
tends to exhibit structural features characteristic of the instrument and its physical 
properties. 

Looking first at the general characteristics of the different microwave profiles, we 
observe that all of the rain rate profiles for KWAJ and MELB are well correlated and 
exhibit good agreement with GV across the nominal mid-range of each plot (i.e., 1 and 10 
mm hr" 1 ). The most significant deviations relative to GV occur at the lowest and highest 
rain rates. The microwave profiles, when considered as a single group, show a distinct 
tendency to overestimate the low rain rates less than 1.0 mm/hr and underestimate the 
high rain rates greater than 10 mm/hr. This pattern results in a crossing of the one-to-one 
line shown in Figs. 10 and 1 1. In evaluating this trend it is important to keep in mind that 
the seesaw relationship between the lower and upper rain rate regimes is fundamentally 
asymmetric in that the total number of samples at the low and high end differ by about 
three orders of magnitude. There is subsequently considerably more sampling of the 
lower rain rates, while the higher rain rates above 10 mm hr" 1 are more rare at the 
0.25°scale. 

The inter-sensor dispersion is largest for observed rain rates below 1.0 mm hr" 1 . 
This region of the profile approaches the sensitivity threshold of each sensor and its 
capacity to detect and quantitatively measure light rain rates. At the high end of each 
profile, we attribute the crossing pattern in each profile, relative to the GV rain rate 
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continuum, to the well-known saturation of the brightness temperatures at high rain rates 
and beam filling in the low frequency channels over the oceans. 

We expect beam filling to be more prominent at the higher rain rates in 
association with smaller scale convection where strong rainfall gradients are more likely 
to occur. Looking at Table 2, it is seen that TMI and AMSR-E have about the same FOV 
compared to the SSM/I, which is considerably larger, but that the mean rain rates are 
commensurate. We consequently do not observe significant inter-sensor differences that 
can be attributed to differences in the FOV. However, examining the scatter diagrams in 
Figs. 8 and 9 for the KWAJ and MELB ocean cases, we clearly observe more scatter at 
the higher rain rates in the SSM/I case. The TMI and AMSR-E, on other hand, exhibit a 
much tighter configuration of points around the one-to-one line. Given the similarities in 
the instrument characteristics, it we attribute these differences observed in the 
instantaneous matched data can be attributed to differences of the FOV between the 
senors. Inter-sensor differences in the FOV, subsequently, appear to be affecting the 
variance in the instantaneous estimates, but not the statistical mean, which in this case 
represents four years of matched data (see Figs. 8 and 9 for sample sizes) 

Kummerow (2001) notes that one of the significant improvements to GPROF rain 
algorithm in version 6 is that it accounts for the beam filling in the lower frequency 
channels by statistically estimating the fraction of convection in the footprint as 
determined from the high frequency channels (85.5 for TMI and 89 for AMSR-E). It is 
expected that this correction should improve the statistical estimate of the mean at the 
higher rain rates, but would not necessarily lesson the random errors (variance) at 
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instantaneous scales, which is what we observe when comparing Figs. 1 1 and 12 with 
Figs. 8 and 9. 

In the first case of KWAJ shown in Fig. 10, AMSR-E and the TMI show the best 
agreement with GV at the lowest rates. Each sensor appears highly correlated with GV, 
even at the lowest rain rates. This observation is further supported by the PDFs for TMI 
and AMSR shown in Figs. 4. Both sensors exhibit impressive statistical agreement with 
GV at the low rain rates. It should be emphasized, however, that the rain rate profiles are 
constructed from one-to-one matching of the satellite with GV in time and space; the 
PDFs, on the other hand, are computed independently for the satellite and GV from the 
matched data. 

TMI and AMSR were the only microwave sensors that exhibited a negative total 
rainfall difference over the four- year period for rain rates less than 1.0 mm hr" 1 . From 
Table 2 it is seen that at KWAJ, TMI and AMSR had -14.0% and -16.5% biases relative 
to GV, respectively while in the case of MELB (ocean), the biases were only -5.3% for 
AMSR-E and -1 1.3% for the TMI. Some of these differences between KWAJ and 
MELB may be attributable to differences in the diurnal cycles of the two regions. MELB 
exhibits a modified oceanic diurnal cycle, with small maximum amplitude in the late 
afternoon. TRMM, as noted earlier temporally samples the entire diurnal cycle, while 
AMSR-E flies over each site twice per day at about the same times as shown in Figs 3a 
and 3b. 

SSM/I sensors F13, F14 and F15 also show good agreement with GV for KWAJ. 
Overall biases for each of the three SSM/I sensors fall within a tight range and exhibit 
similar features at the low and high end. SSM/I sensors performed even better for the 
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MELB ocean case with biases ranging between 0.5 and -4.0 percent. The AMSU-B 
sensors N15, N16 and N17, in turn, display properties common to the AMSU-B group, 
overestimating the rainfall at rain rates less than 1 mm hr" 1 more than the other sensors. 
This result was expected because the AMSU-B rain rates are inferred from an ice 
scattering algorithm. Consequently, the observed rain signature is less correlated with 
surface rainfall (Spencer et al. 1989). At rain rates greater than 10 mm hr' 1 , the N16 and 
N17 profiles clearly underestimated the rain rates more than the other sensors, resulting 
negative biases of -24.5 and -18.4 percent, respectively. The AMSU-B sensors for MELB 
ocean show more intra-group variance, with biases ranging between -28.9 and -13.9 
percent. 

For MELB, we observe more inter-sensor dispersion in the land and coast cases. 
Some of this additional variance is attributable to the smaller sample sizes available for 
each terrain category, but algorithmic uncertainties are also greater due to the dependence 
of inferred rain rates on passive scattering signatures. For MELB land, inter-sensor biases 
range from -18.3 for N17 to 25.5 for F13, and for MELB coast, the satellite biases range 
from -42.9 to 48.4. Inter-sensor dispersion is greatest at the low end of the rain rate 
profile. The land and coast profiles for TMI and AMSR show the best agreement and 
highest overall correlation with GV. The three AMSU-B sensors reveal the greatest 
amount of dispersion within the group. In the case of AMSU-B, some of this variability 
maybe partly ascribed to differences in the diurnal sampling. SSM/I sensors are fairly 
well correlated with GV over land and coast but tend to exceed GV across most the 
dynamic range, resulting in land biases of 25.5, 16.0 and 13.1 percent and coast biases of 
48.4, 71.1 and 44.4 percent for F13, F14 and F15, respectively. 
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e. Two-dimensional Heidke Skill Scores (HSS) 

A common metric used to assess the skill of a given estimate (observation) to 
measure a predicted rain rate is the Heidke Skill Score (HSS); however, as noted by 
Connor and Petty (1998), “ In this context, HSS only yields information concerning the 
algorithm ’s ability to mimic the radar ’s delineation of the lightest possible surface 
precipitation — something that radars themselves are not particularly adept at, especially 
at greater distances ... Furthermore, since area coverage by light precipitation is a 
strong function of threshold rain rate, minor differences in the minimum rain rate 
detectable by each data source can severely degrade the apparent skill of an otherwise 
robust algorithm. We therefore view HSS as a nearly meaningless measure of algorithm 
performance when used in this way.’’'’ 

Table 5 shows a generalized 2x2 contingency table, where in this study, the 
satellite represents the “Observed” values, while GV estimates serve as the 
“Predicted”values. The four results illustrate the observed values “Hits”, “Misses”, 
“False Alarms”, and “Correct Rejections.” These values can then be used to compute a 
number of useful metrics, including: bias, proportion correct, probability of detection, 
false alarm rates, probability of false detection and other metrics. We use the definition 
the HSS as that of Connor and Petty (1998), and follow their lead by constructing 
multiple contingency tables, using separate rain-rate thresholds for both the validation 
and the satellite estimates. The resultant array is then used to generate the plots shown in 
Figs. 12-15. For this study, we chose rain rates for both estimates ranging from 0.1-20 
mm hr" 1 , in increments of 0.1 mm hr" 1 . Equation 3 provides a definition of HSS: 
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3 ), 


= 2 , 2{HC ~ FM) 

F 2 + M 2 + 2HC + (F + M)(H + C) 

where H (Hits), M (Misses), F (False Alarms) and C (Correct rejections) are defined in 
Table 5. 

These plots provide significant information regarding the bias and correlation of 
the two estimates, all as a function of rain rate. If the line of maximum HSS is 
above/along/below the 1:1 line, then the estimate is high-/non-/low-biased, respectively. 
Also, the larger the gradient of the HSS values around the line of maximum HSS are, the 
higher the correlations are. 

Fig. 12 shows the Two-Dimensional HSS (HSS2D) for KWAJ for all of the 
satellite estimates evaluated in this study. TMI and AMSR clearly show the highest skill 
at all rain rates, with maximum HSS of greater than 0.8 for rain rates up to about 2 mm 
hr" 1 , and generally better at all estimates within 0-15 mm hr' 1 , although there is a negative 
bias, which is exacerbated at the higher rain rates, and is also shown in Table 3. TMI 
shows similar skill as AMSR-E at rain rates below about 5 mm h" 1 . The various SSM/I 
estimates are similar, but clearly show less skill than the AMSR-E and TRMM estimates. 
There is a secondary maximum of skill for the SSM/I estimates for rain rates between 
about 12-20 mm hr" 1 . Of the three AMSU-B estimates, N15 and N17 show the greatest 
skill at KWAJ, while N16 shows the least skill, and strong bias above about 4 mm hr" 1 , 
illustrated by the axis of maximum HSS below the 1 : 1 line. 

Figure 13 shows, for each validation rain rate, the maximum HSS at KWAJ, again 
illustrating that AMSR-E shows the best skill at all rain rates, followed by TMI, SSM/I 
and AMSU. The maximum HSS values TMI and AMSR are about 0.2 higher than all 
other estimates, for all rain rates up to about 15 mm hr" 1 . The SSM/I skills are in general 


30 



higher than the AMSU-B skills by about 0.1, with some increased skill of about +0.2 
(relative to AMSU) for rain rates greater than 10 mm hr" 1 . Connor and Petty (1998) 
noted that these types of plots make it possible to evaluate and objectively inter-compare, 
the maximum potential skill of competing algorithms, irrespective of algorithm 
calibration differences. Further, they correctly state that there is no built-in assumption 
of linearity in the relationship between the satellite estimate and the validation data, as 
would be inferred from simple scatter-plots. 

Fig. 14a shows the HSS2D for MELB over ocean areas only. Once again, the TMI 
and AMSR skills are quite high, with maximum HSS values of about 0.8 in the 1-2 mm 
hr" 1 range. Also, there is a clear negative bias in both TMI and AMSR estimates, which is 
exacerbated at rain rates greater than 10 mm hr" 1 . Although the biases of the SSM/I 
estimates tend to be smaller than the AMSR-E biases, their skills are lower. AMSU-B 
shows the lowest skill, again, over ocean areas at MELB, with little or no skill in 
observing rain rates beyond about 10 mm hr" 1 . 

Over land, shown in Fig. 14b, the AMSR-E skills are not as high as over ocean, but 
are unbiased. TMI actually shows slight increase in skill, relative to AMSR-E. The 
SSM/I shows slightly lower skills than AMSR-E, but biases are near zero for rain rates 
less than 10 mm hr" 1 . AMSU-B shows the lowest skills at all rain rates, with a negative 
bias (i.e. AMSU-B less than GV). Over coast, the AMSR-E and SSM/I estimates show 
skills on the order of 0.5 to 0.8 for rain rates less than 10- 15 mm hr' 1 ; however, the 
AMSR-E estimates are negatively biased (-8.2%), while all of the SSM/I estimates are 
positively biased (+44.4% - 71.1%). Given that AMSR-E, TMI and SSM/I all use the 
GPROF algorithm, we are not sure how to assess the discrepancy between their 
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performances over coastal areas, but suggest that algorithm developers investigate 
possible reasons and incorporate the apparent improvements into their algorithms. 

Finally, over the entire GV domain at MELB, shown in Fig. 14d, AMSR-E shows the 
highest skills, with a slight negative bias, while SSM/I show somewhat reduced skill, but 
either near-unity or slightly positive biases, and AMSU-B show the lowest skill and are 
strongly negatively biased. 

Figure 15 shows the maximum HSS for each validation rain bin. Over ocean, both TMI 
and AMSR show significantly better skill, at all rain rates than SSM/I and AMSU-B 
estimates. Over land, differences between the HSS maxima between AMSR-E, TMI and 
other estimates is not as large, but still show increased skills over all, except the highest 
rain rates, where F15 show some increased skill at rain rates greater than 17 mm hr" 1 . 

5. Summary and Conclusions 

In this study, four years (2003-2006) of instantaneous radar rain estimates 
obtained from Tropical Rainfall Measuring Mission (TRMM) Ground Validation (GV) 
sites at Kwajalein, Republic of the Marshall Islands (KWAJ) and Melbourne, Florida 
(MELB) were used to assess the relative performance of satellite precipitation estimates 
from seven polar-orbiting satellites and the TMI on board TRMM. Instantaneous rain 
rates derived from each microwave sensor on board the different satellites were matched 
to the GV estimates in time and space at a resolution of 0.25 degrees. The study evaluates 
the measurement and error characteristics of the various satellite estimates through inter- 
comparisons with GV radar estimates. The GV rain observations provided an empirical 
ground-based reference for assessing the relative performance of each sensor and sensor 
class. 
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All eight satellites compared well with GV at KWAJ and MELB when inter- 
compared on monthly scales, especially over the ocean where all of the available rain 
infonnation was utilized. The microwave sensors were more prone to overestimates over 
land and coast, and not unexpectedly, they performed the worst in the case of MELB 
coast. AMSR-E and the TMI generally performed the best over all of terrain types, 
though their biases over the ocean were commensurate with the other satellites. TMI and 
AMSR also showed the best correlation with GV and displayed the highest skills in 
observing GV rain rates over the full dynamic range of the observations. Table 3 
illustrates that over ocean at KWAJ, all of the satellite estimates showed negative biases 
on the order of -10% to -25%. 

Table 4 revealed that over ocean areas at MELB, the biases were also generally 
negative, ranging from +0.5% (FI 5) to -29% (N17). Over land at MELB, there was 
considerable more spread in the sign and magnitude of the biases, with a +25.5% positive 
bias for F13, and -18% bias for N17. Also, over land at MELB, AMSR-E displayed a 
very small bias of -1.2%, while TMI showed a positive bias of 12.2%. We infer that the 
differences in these biases are due to the following: 1) the TRMM orbit allows a full 
sample of the diurnal cycle, especially over the large temporal scales employed in this 
study; 2) minor variations in the instrumentation of the different platforms; and 3) 
variations in the algorithms used to retrieve the rain rates, often referred to as 
“versionitis.” 

Over coastal areas at MELB (Table 4), there was considerable deviation in the 
biases, with SSM/I showing large positive biases ranging from +44% to +71%, as well 
the rather inconsistent AMSU-B estimates, which range from -43% to +0.5%. TMI and 


33 



AMSR-E both showed relative small negative biases of -12.9% and -8.2%, respectively. 
Looking at the entire domain, irrespective of the terrain type, the biases range from - 
31.8% (N17) to +33.5% (F14), with TMI and AMSR showing remarkably small biases of 
-4.2% and -5.5%. 

It is important to note that assessing the biases alone is not sufficient to detennine 
which satellite estimates fare best, especially since the GV estimates themselves may 
incur some bias. Further, a simple comparison of the PDFs can also be misleading, 
because the information relating to the time-space matching is lost in the statistical 
construction of the PDF. Hence, analysis of scatter-plots, simple statistics, and the two 
dimensional skill scores, as was done in this study, is needed to provide a more complete 
picture of how a satellite estimate is performing, especially with respect to the 
observation (i.e. detection) surface precipitation from space. 

The rain rate profiles generated in Figs. 10 and 11 revealed that only TMI and 
AMSR-E performed well in estimating GV rain rates less than 1.0 mm hr" 1 , while the 
other satellites tended to overestimate the rain rates in this region. This result is important 
because the sampling was extremely high in that region of each profile. Two of the 
AMSU-B estimates (N15 and N16) performed the worst in overestimating the low rain 
rates, while N17 underestimated the high rain rates by more the any of the other satellites. 
It is noteworthy that over land and coast there was more overall dispersion between the 
different satellites. The profiles for each sensor group, nonetheless, tended to display 
similar characteristics. 

Probability distributions for the various satellites indicate that both TMI and 
AMSR-E are significantly better, and more highly correlated with GV estimates than the 
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SSM/I retrievals, and to a larger extent the AMSU-B estimates. Ad hoc adjustment of the 
AMSU-B estimates of light rain rates (approximately 1mm hr" 1 ) over ocean, in particular, 
provides for physically implausible distributions, and caution should be taken when these 
data are used. Two-dimensional Heidke Skill Score plots show also that both TMI and 
AMSR-E are significantly better skilled at detecting surface precipitation, providing less 
biased, more highly correlated, and more linear comparisons than SSM/I and 
significantly more so than AMSU-B estimates, especially over ocean. Over land areas, 
the differences between the skills of the various estimates is smaller, but we find that the 
TMI estimates be superior, followed by AMSR-E, SSM/I and AMSU-B. 

These results signal developers of global rainfall products, such as the TRMM 
Multi-Satellite Precipitation Analysis (TMPA), that care must be taken when 
incorporating data from these input satellite estimates in order to provide the highest 
quality estimates when attempting to merge the data from external sources. For example, 
as is currently done with the TMPA product, if two or more estimates are available in a 
given 3 -hour, 0.25° grid, the two estimates would be averaged. We would suggest 
instead that the contributions be weighted by quality in order to provide the most robust 
estimates. 
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center of each image and range rings at 50 km, 100 km and 150 km are also shown. 

Fig. 2b: Same as Fig. 2a, except for MELB 
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KWAJ using GV as an empirical reference. Satellite rain rates were binned and sorted 
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along the GV rain rate continuum. 


Fig. 11: Rain rate profiles for AMSR, FI 3, F14, FI 5, N15, N16 and N 17 generated from 
matched GV-satellite data set for MELB using GV as an empirical reference. Satellite 
rain rates were binned and sorted along the GV rain rate continuum. The matched data 
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Fig. 12: Two-dimensional Heidke Skill Score plots for the various satellite and GV 
estimates. The line through the contours represents the maximum HSS for a given GV 
rain rate. 

Fig. 13: Maximum Heidke Skill Scores for a given GV rain rate at KWAJ for each of the 
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Fig. 14a: Same as Fig. 12, except over ocean areas at MELB. 

Fig. 14b: Same as Fig. 12, except for land areas over MELB. 

Fig. 14c: Same as Fig. 12, except for coastal areas over MELB. 

Fig. 14d: Same as Fig. 12, except for the entire GV domain over MELB. 
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Fig 15: Maximum Heidke Skill Score for a given GV rain rate at MELB for the various 
satellites. The panels show these scores over “Ocean”, “Land”, “Coast” and “All” in 
the top-left, top-right, bottom-left and bottom-right panels, respectively. 


45 



Table 1. : List of available frequencies (v), polarizations (a), and beam widths (6) of the 
various channels for the satellites assessed in this study. V and H represent vertical and 
horizontal polarizations, respectively. Column 1 provides the name of the respective 

satellite platform. 


Sensor 

Farm 

Channels (GHz) 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

AMSR-E 

V 

6.9 

6.9 

10.7 

10.7 

18.7 

18.7 

23.8 

23.8 

36.5 

36.5 

89.0 

89.0 


a 

V 

H 

V 

H 

V 

H 

V 

H 

V 

H 

V 

H 


6 

2.2 

2.2 

1.5 

1.5 

0.8 

0.8 

0.92 

0.92 

0.42 

0.42 

0.18 

0.18 


V 

19.35 

19.35 

22.3 

37.0 

37.0 

85.5 

85.5 






SSM/I 

a 

V 

H 

V 

V 

H 

V 

H 







6 

1.86 

1.88 

1.6 

1.0 

1.0 

0.41 

0.42 






AMSU-A, B* 

V 

23.8 

31.4 

89.0 

150 

183.3 1 

183.3 2 








a 

V 

V 

V 

V 

V 

V 

V 







e 

3.3 

3.3 

1.1 

1.1 

1.1 

1.1 

1.1 







V 

10.7 

10.7 

19.35 

19.35 

21.3 

37.0 

37.0 

85.5 

85.5 




TMI 

a 

V 

H 

V 

H 

V 

V 

H 

V 

H 





e 

3.68 

3.75 

1.9 

1.88 

1.7 

1.0 

1.0 

0.42 

0.43 





*AMSU-B channels at 89.0, 150, and 183 GHz correspond to channels numbers 16-20 

1. 183.3 ±1.00 GHz; 

2. 183.3 ±3.00 GHz; 

3. 183.3 ±7.00 GHz 
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Table 2: Average altitudes and footprint sizes for various channels of the satellite 

platforms utilized in this study. 


Rain Sensor 

Satellite 

Altitude 

(km) 

23 GHz 
(km 2 ) 

37 GHz 
(km 2 ) 

89.0 GHz 
(km 2 ) 

AMSR-E 

Aqua 

705 

31 x 18 

14x8 

4x6 

TMI 

TRMM 

402 

27.2 x 16.5 

16x9.7 

6.9 x 4.2 

SSM/I 

F13, F14, F15 

830 

59.7x39.6 

35.4x29.2 

15 x 13 

AMSU 

N15, N16, N16 

830 

n/a 

n/a 

15 x 15 
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Table 3: General statistics derived from the period 2003-2006, showing comparisons 
between TRMM GV estimates at KWAJ versus the various satellites estimates assessed in 
this study. The columns, from left to right, correspond to site, satellite, GV mean rain 
rate (mm hr), satellite mean rain rate (mm hr 1 ), regression equation intercept and 
slope, correlation, and bias. The bias, expressed in percentage is defined via the 
following: Bias = 100%*(Sat_mean - G Vjnean)/G Vjnean, so that a negative bias 
indicates a satellite underestimate, relative to GV. 


Site 

Sat 

Mask 

GV 

Mean 

Sat 

Mean 

Slope 

Corr 

Bias 

KWAJ 

F13 

Ocean 

0.231 

0.196 

0.74 

0.67 

-15.0 

KWAJ 

F14 

Ocean 

0.238 

0.205 

0.78 

0.67 

-13.9 

KWAJ 

F15 

Ocean 

0.232 

0.203 

0.68 

0.63 

-12.5 

KWAJ 

N15 

Ocean 

0.241 

0.21 

0.65 

0.66 

-12.8 

KWAJ 

N16 

Ocean 

0.266 

0.201 

0.49 

0.62 

-24.5 

KWAJ 

N17 

Ocean 

0.242 

0.198 

0.57 

0.65 

-18.4 

KWAJ 

AM SR 

Ocean 

0.231 

0.193 

0.75 

0.89 

-16.5 

KWAJ 

TMI 

Ocean 

0.245 

0.211 

0.79 

0.86 

-14.0 
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Table 4: Same as Table 3, except for MELB. Also, the statistics are sub-classified by 
terrain type (Ocean, Land, Coast and All). 


Site 

Sat 

Mask 

GV 

Mean 

Sat 

Mean 

Slope 

Correlation 

Bias 

MELB 

F13 

Ocean 

0.123 

0.12 

0.65 

0.64 

-2.5 

MELB 

F14 

Ocean 

0.134 

0.128 

0.63 

0.63 

-4.0 

MELB 

F15 

Ocean 

0.144 

0.144 

0.7 

0.69 

0.5 

MELB 

N15 

Ocean 

0.143 

0.123 

0.54 

0.56 

-13.9 

MELB 

N16 

Ocean 

0.136 

0.111 

0.53 

0.51 

-18.1 

MELB 

N17 

Ocean 

0.132 

0.094 

0.47 

0.53 

-28.9 

MELB 

AM SR 

Ocean 

0.153 

0.145 

0.69 

0.83 

-5.3 

MELB 

TMI 

Ocean 

0.131 

0.116 

0.65 

0.83 

-11.3 

MELB 

F13 

Land 

0.169 

0.212 

0.72 

0.64 

25.5 

MELB 

FI 4 

Land 

0.11 

0.127 

0.69 

0.67 

16.0 

MELB 

F15 

Land 

0.114 

0.128 

0.68 

0.67 

13.1 

MELB 

N15 

Land 

0.181 

0.212 

0.52 

0.5 

17.1 

MELB 

N16 

Land 

0.18 

0.166 

0.46 

0.49 

-7.7 

MELB 

N17 

Land 

0.094 

0.077 

0.45 

0.57 

-18.3 

MELB 

AMSR 

Land 

0.164 

0.163 

0.71 

0.72 

-1.2 

MELB 

TMI 

Land 

0.14 

0.157 

0.79 

0.73 

12.3 

MELB 

F13 

Coast 

0.125 

0.185 

0.9 

0.63 

48.4 

MELB 

F14 

Coast 

0.114 

0.195 

1.04 

0.65 

71.1 

MELB 

F15 

Coast 

0.118 

0.17 

0.91 

0.68 

44.4 

MELB 

N15 

Coast 

0.148 

0.148 

0.48 

0.48 

0.5 

MELB 

N16 

Coast 

0.166 

0.143 

0.44 

0.48 

-14.2 

MELB 

N17 

Coast 

0.094 

0.054 

0.36 

0.53 

-42.9 

MELB 

AMSR 

Coast 

0.172 

0.158 

0.65 

0.67 

-8.2 

MELB 

TMI 

Coast 

0.135 

0.118 

0.66 

0.74 

-12.9 

MELB 

F13 

All 

0.137 

0.175 

0.77 

0.63 

28.1 

MELB 

F14 

All 

0.118 

0.158 

0.82 

0.63 

33.5 

MELB 

F15 

All 

0.124 

0.151 

0.78 

0.67 

22.5 

MELB 

N15 

All 

0.156 

0.16 

0.51 

0.5 

2.5 

MELB 

N16 

All 

0.162 

0.141 

0.46 

0.49 

-13.0 

MELB 

N17 

All 

0.104 

0.071 

0.42 

0.54 

-31.8 

MELB 

AMSR 

All 

0.165 

0.156 

0.68 

0.72 

-5.5 

MELB 

TMI 

All 

0.136 

0.13 

0.7 

0.74 

-4.2 
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Table 5: Contingency table construction used for calculating the Heidke Skill Score and 
other metrics. In our study, satellite estimates are '‘Observed”, while validation 

estimates are “ Predicted . ” 



Observed Yes 

Observed No 

Predicted Yes 

A (Hits) 

C (False Alarms) 

Predicted No 

B (Misses) 

D (Correct Rejections 
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KWAJ GV Network 


T : T 



6.50 


• WSR-88D ■ KWA gauges 

Fig. la: Illustration of the gauge and radar networks for the GV network at KWAJ. 
Range rings at 50, 100, 150 and 200 km are also shown. Note that for this study, all 
usable GV data was restricted to the first 100 km from each respective radar site. KWAJ 
has only seven gauge sites; however, each site contains two or more gauges to improve 
reliability and uncertainty of the measured rain rates. 
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MELB GV Network 



• WSR-88D • KSC gauges o STJ gauges • SFL gauges 

Fig. lb: Same as Fig. la, except for MELB. There are three gauge networks in Florida: 
St. John ’.s' River Water Management District (STJ), and the South Florida Water 
Management District (SFL). Both of these networks are operated by the state of Florida. 
A NASA-owned network is located on Cape Canaveral at NASA Kennedy Space Center. 
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GPROF Terrain Mask for KWAJ 



Fig. 2a: Terrain masks for the KWAJ GV site. Each 0.25° grid box is designated as ocean 
(3), land (0), coastal-water (4), or coastal land (5). Both coastal-land and coastal-water 
are treated together as “coast” in this study. The respective GV radar is located at the 
center of each image and range rings at 50 km, 100 km and 150 km are also shown. 
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GPROF Terrain Mask for MELB 


Fig. 2b: Same as Fig. 2a, except for MELB 
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KWAJ Diurnal Cycle (2000-2004) 



0 5 10 15 20 

Hour of the Day 

Fig. 3a: Diurnal cycle of hourly conditional mean rain rate, derived from seven years 
( 2000-2006 ) of GV data at HSTN. Superimposed are colored symbols showing the local 
overpass times of each of the polar-orbiting satellites assessed in this study. 
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Fig. 3b: Same as Fig. 2a, except for MELB. 
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Fig. 4: Probability density functions (PDF) of rain rates for TRMM GV and the various 
satellite retrievals at KWAJ. The dashed and dotted lines represent the PDF by 
occurrence PDF C =PDF(R) for GV and satellite, respectively, while the solid and dash- 
dotted lines represent the PDF by volume PDF V =[R*PDF(R)] for GV and Satellite, 
respectively 
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Fig. 5 a: Probability density functions (PDF) of rain rates for TRMM GV and the various 
satellite retrievals at MELB over ocean areas only. The dashed and dotted lines 
represent the PDF by occurrence PDF C =(PDF(R)J for GV and satellite respectively, 
while the solid and dash-dotted line represent the PDF by volume PDF V =[R*PDF(R)J. 
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Fig 5 b: Same as Fig. 5a, except over land areas only at MELB. 
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Fig. 5c: Same as Fig. 5a, except for coastal areas only at MELB. 
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Fig. 5d: Same as Fig. 5a, except for the entire GV domain at MELB. 
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Fig. 6: Monthly rain computed estimates for KWAJ for each of the seven space-borne 
microwave estimates. Rain estimates only integrated matched GV-satellite rain rates 
during satellite overpasses. 
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Fig. 7a: Same as Fig. 6, except for over ocean areas at MELB. 
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Fig. 7b: Same as Fig. 6, except for over land areas only at MELB. 
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Fig. 7c: Same as Fig. 6, except for over coastal areas only at MELB. 
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Fig. 7d: Same as Fig. 6, except for over the entire GV domain at MELB. 
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Fig 8: Scatter plots of instantaneous satellite and GV rain rates for KWAJ/Ocean for the 
period 2003-2006. 
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Fig 9a: Scatter plots of instantaneous satellite and GV rain rates for MELB over ocean 
areas only, for the period 2003-2006. 
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Fig 9b: Same as Fig. 9a but over land areas only. 
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Fig 9c: Same as Fig. 9a but over coastal areas only. 
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Fig. 9d: Same as Fig. 9a, but over the entire GV domain at MELB. 
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Fig. 10: Rain rate profiles for AMSR, FI 3, F14, FI 5, N15, N16 and N 17 generated for 
KWAJ using GV as an empirical reference. Satellite rain rates were binned and sorted 
along the GV rain rate continuum. 
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Fig. 11: Rain rate profiles for AMSR, FIS, F14, F15, N15, N16 and N 17 generated from 
matched GV-satellite data set for MELB using GV as an empirical reference. Satellite 
rain rates were binned and sorted along the GV rain rate continuum. The matched data 
was further stratified according to Ocean (top left), Land (top right), Coast (bottom left) 
and All (bottom right). 


73 




KWAJ/F13 Ocean 




KWAJ/F15 Ocean 


0 5 10 15 20 

GV Rain Rate [mm/hrj 


5 10 15 20 

GV Rain Rate [mm/hr] 


KWAJ/F14 Ocean 

20 


0 

0 5 10 15 20 

GV Rain Rate [mm /hr] 






KWAJ/N15 Ocean 

20 


0 

0 5 10 15 20 

GV Rain Rate [mm/hr] 


KWAJ/N16 Ocean 

20 


0 5 10 15 20 

GV Rain Rate [mm/hr] 


KWAJ/N17 Ocean 

20 


0 5 10 15 20 

GV Rain Rate [mm/hr] 




KWAJ/AMSR Ocean 

20 


0 

0 5 10 15 20 

GV Rain Rate [mm/hr] 


KWAJ/TMI Ocean 

20 


0 

0 5 10 15 20 

GV Rain Rate [mm/hr] 


Heidke Skill Score 

■ U 


Fig. 12: Two-dimensional Fleidke Skill Score plots for the various satellite and GV 
estimates. The line through the contours represents the maximum HSS for a given GV 
rain rate. 
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Fig. 13: Maximum Heidke Skill Scores for a given GV rain rate at KWAJ for each of the 
various satellites. Line colors specify the particular satellite estimate. 
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Fig. 14a: Same as Fig. 12, except over ocean areas at MELB. 
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Fig. 14b: Same as Fig. 12, except for land areas over MELB. 
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Fig. 14c: Same as Fig. 12, except for coastal areas over MELB. 
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Fig. 14d: Same as Fig. 12, except for the entire GV domain over MELB. 
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Fig 15: Maximum Heidke Skill Score for a given GV rain rate at MELB for the various 
satellites. The panels show these scores over “Ocean”, “Land”, “Coast” and “AH” in 
the top-ieft, top-right, bottom-left and bottom-right panels, respectively. 
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POPULAR SUMMARY 


Assessing the Relative Performance of Microwave-Based Satellite Rain Rate 
Retrievals Using TRMM Ground Validation Data 

David B. Wolff 1 ' 2 and Brad L. Fisher 1 ' 2 

1 Science Systems & Applications, Inc, Lanham, Maryland 

2 NASA Goddard Space Flight Center, Greenbelt, Maryland 

Space-borne microwave sensors provide critical rain information used in several 
global multi-satellite rain products, which in turn are used for a variety of important 
studies, including landslide forecasting, flash flood warning, data assimilation, 
climate studies, and validation of model forecasts of precipitation. This study 
employs four years (2003-2006) of satellite data to assess the relative performance 
and skill of SSM/I (F13, F14 and F15), AMSU-B (N15, N16 and N17), AMSR-E (Aqua) 
and the TRMM Microwave Imager (TMI) in estimating surface rainfall based on 
direct instantaneous comparisons with ground-based rain estimates from Tropical 
Rainfall Measuring Mission (TRMM) Ground Validation (GV) sites at Kwajalein, 
Republic of the Marshall Islands (KWAJ) and Melbourne, Florida (MELB). The 
relative performance of each of these satellite estimates is examined via 
comparisons with space- and time-coincident GV radar-based rain rate estimates. 
Because underlying surface terrain is known to affect the relative performance of 
the satellite algorithms, the data for MELB was further stratified into ocean, land 
and coast categories using a 0.25° terrain mask. 

Of all the satellite estimates compared in this study, TMI and AMSR-E exhibited 
considerably higher correlations and skills in estimating/observing surface 
precipitation. While SSM/I and AMSU-B exhibited lower correlations and skills for 
each of the different terrain categories, the SSM/I absolute biases trended slightly 
lower than AMSRE over ocean, where the observations from both emission and 
scattering channels were used in the retrievals. AMSU-B exhibited the least skill 
relative to GV in all of the relevant statistical categories, and an anomalous spike 
was observed in the probability distribution functions near 1.0 mm hr-1. This 
statistical artifact appears to be related to attempts by algorithm developers to 
include some lighter rain rates, not easily detectable by its scatter-only frequencies. 
AMSU-B, however, agreed well with GV when the matching data was analyzed on 
monthly scales. These results signal developers of global rainfall products, such as 
the TRMM Multi-Satellite Precipitation Analysis (TMPA), and the Climate Data 
Center’s Morphing (CMORPH) technique, that care must be taken when 
incorporating data from these input satellite estimates in order to provide the 
highest quality estimates in their products. 



